Joint Action Learners in Competitive Stochastic Games

This thesis investigates the design of adaptive utility maximizing software agents for competitive multi-agent settings. The focus is on evaluating the theoretical and empirical performance of Joint Action Learners (JALs) in settings modeled as stochastic games. JALs extend the well-studied Q-learning algorithm. A previously introduced JAL optimizes with respect to stationary or convergent opponents and outperforms various other multi-agent learning algorithms from the literature. However, its deterministic best-response dynamics do not allow it to perform well in settings where non-determinism is required. A new JAL is presented which overcomes this limitation. Non-determinism is achieved through a randomized action selection mechanism discussed in the game theory community. The analysis of JALs is conducted with respect to a new set of evaluation criteria for self-interested agents. Further research is required before all criteria could be met reliably. In addition, some learning desiderata prove impossible to achieve in settings where the rewards of the opponents are not observable.